Automatic Detection of Human Faces in Natural Scene Images by Use of a Skin Color Model and of Invariant Moments

We use a skin color model based on the Mahalanobis metric and a shape analysis based on invariant moments to automatically detect and locate human faces in two-dimensional natural scene images. First, color segmentation of an input image is performed by thresholding in a perceptually plausible hue-saturation color space where the effects of the variability of human skin color and the dependency of chrominance on changes in illumination are reduced. We then group regions of the resulting binary image which have been classified as face candidates into clusters of connected pixels. Performing median filtering on the image and discarding the smallest remaining clusters ensures that only a small number of clusters will be used for further analysis. Fully translation-, scaleand in-plane rotation-invariant moments are calculated for each remaining cluster. Finally, in order to distinguish faces from distractors, a multilayer perceptron neural network is used with the invariant moments as the input vector. Supervised learning of the network is implemented with the backpropagation algorithm, at first for frontal views of faces. Preliminary results show the efficiency of the combination of color segmentation and of invariant moments in detecting faces with a large variety of poses and against relatively complex backgrounds.